Integrating Graphics Into Video Image-Based Camera Tracking and Filtering

In virtual studios, most of the setting is not constructed in reality but generated synthetically by a graphics computer and mixed into the camera image. A convincing mixing requires very precise coordination of the real camera with the graphics pipeline, especially in the case of camera moves. This process is called camera tracking. The paper describes an image-based algorithm for camera tracking. The algorithm uses real world reference points which are identified interactively in the first image and tracked in the image sequence in real-time. Based on these measurements, the camera data is calculated. The most demanding task is the smoothing of measurements and camera data to achieve stable and coherent camera moves. Kalman filtering methods are used to eliminate jitters from the calculated camera data.